1. Technical Field
The invention relates to the production of high resolution images. More particularly, the invention relates to the use of image interpolation to produce a high resolution image that looks as though it was generated directly from a high resolution sampling device.
2. Description of the Prior Art
One goal of image interpolation is to produce a high-resolution image that looks as though it actually came from a high resolution sampling device. Commonly, the acquisition device is a Charge Coupled Device (CCD) which consists of an array of sensors, where each sensor returns a number that is proportional to the average of the light falling on it. To a first order, the CCD array can be modelled as a rectangular grid, where there is negligible spacing between sensors, and each sensor is uniformly sensitive across its area (see FIG. 1). Thus, if the light incident on the two-dimensional array is I(u,v) the output of a single sensor is: ##EQU1## where L.sub.mn is the area occupied by the sensor at the region R.sub.m,n.
A higher resolution CCD camera has more sensors to represent an image of the same size, and the area H.sub.mn of each sensor is smaller (H.sub.mn &lt;L.sub.mn).
If a high resolution array were available, the array's output would be: ##EQU2## where H.sub.mn is the area occupied by the sensor at the region R.sub.m,n.
In scaling the low resolution image x.sub.mn it is desirable to produce a high resolution image y.sub.mn that is as close as possible to y.sub.mn. While y.sub.mn is unknown, it is known that it has the form given in Eq. (2) above. It is desirable to constrain the interpolated image y.sub.mn as at least a possible output of the acquisition device for some image I(u,v), such that: ##EQU3## Thus, from Eqs. (3) and (4) above, the value y.sub.mn is a consistent estimate of y.sub.mn with respect to the low resolution data x.sub.mn and the first order sensor model. In other words, both y.sub.mn and x.sub.mn should be valid outputs of the high and low resolution acquisition devices for some waveform I(u,v).
Note that Eqs. (3) and (4) also imply that if an image is enlarged, and then the image is scaled down, the result of these two consecutive steps generates an image that is identical to the original image.
There are many cells on the surface of a CCD (see FIG. 1), where each cell captures one unit of an image, which is typically referred to as a pixel. The value of the pixel is proportional to the average of the light that shines on that particular area of the CCD. If one was to use a high resolution CCD, for example a CCD having four times the number of pixels of a lower resolution CCD (see FIG. 2), then each one of the four pixels 21, 23, 25, 27 is roughly one quarter of the size of the original large pixels 22. Thus, the average of light that shines on the four smaller pixels 21, 23, 25, 27 is equal to the average of the light that shines on one large pixel 22. Therefore: EQU X.sub.m,n =avg(y.sub.2m,2n,y.sub.2m,2n+1,y.sub.2m+1,2n,y.sub.2m+1,2n+1)(5)
The goal of high resolution image scaling while maintaining data consistency was shown to be achievable with a previous algorithm (see J. Allebach, P. W. Wong, Magnifying Digital Image Using Edge Mapping, U.S. patent application Ser. No. 08/412,640, filed 29 Mar. 1995, which is a continuation of Ser. No. 08/227,765, filed 14 Apr. 1994). The disclosed approach involves an iterative stage (see FIG. 3) that consists of a sensor model 32, and is thus computationally costly. The algorithm takes true sensor data 30 to form a small image and magnifies the small image to produce a large image (the interpolated image 31). This is shown in the figure by the low resolution path 33 and the high resolution path 34.
A feedback path through the sensor model 32 applies the characteristics of the CCD to maintain data consistency. Accordingly, this approach uses the sensor model to scale down the output image. Once the image is scaled down, it is an image that corresponds in the size to the estimated sensor data, which should be the same size image as that of the true sensor data. The algorithm compares these data to determine how much difference there is between the estimated image and the actual image, and then uses the difference to correct the interpolated image with a data correction module 35.
Contrary to the case of scaling one-dimensional band limited signals such as speech, where the goal is to design a filter with response as close as possible to an ideal brickwall filter (see, D. J. Goodman, M. J. Carey, Nine digital filters for decimation and interpolation, IEEE Transaction on Acoustics, Speech, and Signal Processing, vol. ASSP-25, pp. 121-126, April 1977), achieving good frequency response in the traditional sense is not the main goal for image scaling. An explanation of this can be given by observing that if the M-fold interpolator is a very good 1/M low pass filter, then the region of the frequency spectrum .vertline..omega..vertline.&gt;.pi./M in the interpolated image is very close to zero. Thus, the scaled image contains only very low frequency energy, and hence no sharp edges can be reproduced. This correlates well with the accepted wisdom in image scaling that windowed sinc filters generally produce scaled images that look blurred.
A key problem to be addressed then is how to scale images and produce high quality outputs from low resolution acquisition devices. As discussed above, the assumption of band limitedness is a poor assumption for images, and there is no model for image spectra that is sufficiently accurate to allow exact reconstruction from samples. Another way of expressing this is to say that the difficulty in constructing the high frequency component in images is a serious obstacle to image scaling.